The minimax distortion redundancy in noisy source coding

Consider the problem of finite-rate filtering of a discrete memoryless process {Xi}i≥1 based on its noisy observation sequence {Zi}i≥1, which is the output of a Discrete Memoryless Channel (DMC) whose input is {Xi}i≥1. When the distribution of the pairs (Xi, Zi), PX,Z , is known, and for a given distortion measure, the solution to this problem is well known to be given by classical rate-distortion theory upon the introduction of a modified distortion measure. In this work we address the case where PX,Z , rather than being completely specified, is only known to belong to some set Λ. For a fixed encoding rate R we look at the worst case, over all θ ∈ Λ, of the difference between the expected distortion of a given scheme which is not allowed to depend on the active source θ ∈ Λ and the value of the distortion-rate function at R corresponding to the noisy source θ. We study the minimum attainable value achievable by any scheme operating at rate R for this worst-case quantity, denoted by D(Λ, R). Linking between this problem and that of source coding under several distortion measures, we prove a coding theorem for the latter problem and apply it to characterize D(Λ, R) for the case where all members of Λ share the same noisy marginal. For the case of a general Λ, we obtain a single-letter characterization of D(Λ, R) for the finite-alphabet case. This gives, in particular, a necessary and sufficient condition on the set Λ for the existence of a coding scheme which is universally optimal for all members of Λ and characterizes the approximation-estimation tradeoff for statistical modelling of noisy source coding problems. Finally, we obtain D(Λ, R) in closed form for cases where Λ consists of distributions on the (channel) input-output pair of a Bernoulli source corrupted by a Binary Symmetric Channel (BSC). In particular, for the case where Λ consists of two sources: the all-zero source corrupted by a BSC with crossover probability r and the Bernoulli(r) source with a noise-free channel; we find that universality becomes increasingly hard with increasing rate.